ID 139 992 CE on 070 

lUTHOE Johnson^ James |, Trent^ Curtis 

TITLE In Acalysia of Student Evaluation of Course and 

Instructor as an Ixpression of the Student's Values 

ana adult Education Attitudes, 
lUB DATE Apr 77 

HOTl 25p,| Paper presented at the Adult Education Research 

Conference (Minneapolis, Minnesota, April 20*23 , 
1977) 

IDRS EBICE nT-$0.83 HC-$1,67 Plus Postage* 

DISCBIFTOES ^Aflult Iducationi College Facultyi ^Course 

Ivaluationi ^Educational Attitudesi Iducational 
Research; Graduate Studentsi Higher Education; 
^Personal Valuesi ^Student Attitudesi *Student 
Evaluation cf Teacher Performance i Teacher 
Evaluation 



ABSTBACT 

Ihe study determined the influence of students' 
values and adult education attitudes on their evaluation of faculty* 
Eighty^fQUr adult education graduate students, enrolled in four 
sections of an adult learning theory course taught by three different 
instrtictors, completed an evaluation form, a value survey, and an 
adult education attitude scale at the end of the course. Multiple 
regression and path analysis showed that (1) adult education 
attitudes and selected values accounted for 27X of the variance in 
student evaluation of faculty^ (2) adult education attitudes were 
responsifcle for B% of the variance in student evaluation of faculty, 
and (3) values were not significantly related to student evaluation 
of faculty* A path model suggested that values influenced student 
evaluation of faculty through attitudes. The fillings suggested that 
values and attitudes play an important part in student evaluation of 
faculty and should be recognized when interpreting the results of 
sach ratings. (Author) 
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Abstract 

The study determined the influence of students' values and adult 
education attitudes on their evaluation of faculty* Eighty-four adult 
education graduate students coinpleted an evaluation form, a value 
survey, and an adult education attitude scale at the end of an 
adult learning theory course. Multiple regression and path analysis 
showed that (a) adult education attitudes and selected values accounted 
for 27 percent of the variance in student evaluation of faculty, (b) 
adult education attitudes were responsible for 8 percent of the variance 
in student evaluation of faculty and (c) values were not significantly 
related to student evaluation of faculty. A path model suggested 
that values influenced student evaluation of faculty through attitudes. 
The findings suggested that the values and attitudes play an important 
part in student evaluation of faculty and should be recogniEed when 
interpreting the results of such ratings. 
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An Analysis of Student Evaluation of Course and Instructor as an 
Expression of the Student's Vali l es and Adult Education Attitudes 

Why do two individuals evaluate a like experience differently? This 
question, largely unanswered in the literatui led the researchers to 
examine adult education attitudes and values to determine the degree to 
which they could explain why adult education graduate students evaluate 
the same course and instructor differently. 

It V as anticipated that a person *s values and attitudes would in- 
fluence how that individual evaluated a learning experience. It was 
believed that attitudes would have more direct influence or evaluations 
than values. Values were seen as being more central to behavior determi- 
nation than attitudes. Based on this concept ^ it was believed that values 
would only handle those evaluation questions that attitudes could not 
readily resolve. It was believed that attitudes and values were closely 
related and that they each would influence the formation of the other. 

Method 

Eighty-four adult education graduate students enrolled in four 
sections of an adult learniTig theory course participated in the study. The 
sections were taught by three different instructors. The participating 
students completed an adult education attitude scales a value survey, and 
a course and instructor evaluation. The data were collected at the end 
of the fall serimster, 1975^ at North Carolina State University in Raleigh. 
Values were meatmred by the Value Survey » adult education attitudes by the 
Educational Orientation Questionnaire, and the student evaluation of 
faculty by the C ourse and Instructor Evaluation instrument* 
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The Educational Orientation ^uesclonnaire was developed by Herschel 
HadTey (1975) to measure adult education attitudes along an andragogy- 
pedagogy continuum. The instrument contained 60 Likert type items. Test- 
retest reliability was ,89 (Hadley, 1975, p, 88) with a coefflcienC 
alpha of .96 (Hadley, 1973, p. 89)* A high score indicated adult education 
attitudes were closely allied with andragogy. 

The Value Survey developed by Milton Rokeach (1967 i 1973, pp. 26-52) 
conLalns two lists of 18 alphabetically arranged values. Printed on gummed 
labels j one list contains terininal values which are "end states" or "goals'* 
the respondent wants to achieve. The other list contains instrumental 
values which are "moral values" or "means" that the individual has feelings 
about. The respondent moves the gunmed labels until each list is arranged 
in a hierarchy (1^18, with 1 being the most Important and 18 being the 
Joast Importnnt) of Importance to him* A low score IndicateH that a value 
is considered more important by a respondent. 

The Course and Instructor Evaluation instrument was developed by the 
Dapartment of Adult and Conmunity College Education at North Carolina State 
University and is used to evaluate each class taught by the department. 
The instrument contains 28 items of which 26 are Likert type items concerning 
specific aspects of the learning situation* The other two items are "marco" 
in scope representing an overall assessment of the course and the instructor 
respectively* A high score on the Instrument indicates a high rating for 
the course and the instructor. 

Analysis of Data 

Statlsticnl Analysis System (SAS) programs were used to analyse the 
data. Factor analysis (Service, 1972, pp. 2Q1--2Q3) was used to determine 
the best technique of presenting the Course and Instructor Evaluation data. 




D 



3 



Relationships between variables were analyzed using REGR (Service, 1972, 
p. 99) and CORR (Service, 1972, pp. 208-209) SAS procedures. 

Student evaluation of faculty, as measured by the Course and Inst ructor 
Evaluation instrument was the main dependent variable. Course and instructor 
evaluation was analyzed as a single score that represented the sum of six 
mean factor scores identified by the SAS FACTOR procedure (Service, 1972, 
pp. 201-203) . As the dependent variable, course and instructor evaluation 
was compared with the independent variables using the SAS regression (REGR) 
program* 

Values, as measured by the Value Survey , were independent variables. 
The main analysis grouped the 36 values into seven factors identified by 
Rokeach (1973, p- 47)* The 36 values were also analysed with the other 
variables using CORR procedures. The seven value factors were compared 
with the othur variables by means of the SAS REGR and CORR procedures. 

Adult education attitudes, as measured by the Educational Orientation 
Questionnaire , was the other independent variable. Adult education attitudes 
were anlayzed with the variables as a single score which was the sum of 
eight factor scores* The eight factors identified by Hadley (1975, p. 105) 
were used to obtain the factor scores* Again, REGR and CORR procedures 
were used to compare adult education attitudes, values and course and 
instructor evaluation* 

Regression analysis (REGR) was used to identify a model to use as a 
Path diagram to measure possible combined influence of adult education 
attitudes and values on course and instructor evaluation* Regression 
lines were run for adult education attitudes and values to determine model 
components. Once the possible model was identified, all possible regresr 
sion lines wore run using REGR procedure to develop the Path diagram 
(Babble, 1973, pp, 324-327), ft 



DATA ANALYSIS FRAMEWORK 



Student Student Adult Student Evaluation of 

Valua Factors Education Attitudes Course and Instructor 

XI Imnediate versus delayed gradif ication 

X2 Competence versus religious morality 

X3 Self=coiistriction versus self^expansion Y Z 

X4 Social versus personal orientation ^ 

X5 Societal versus family security 

X6 Reipect versus love 

X7 Inner versus other directed 
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The Data Analysis Framework follows the conceptual model outline 
in the beginning of the paper. It is depicted graphically as follows: 



Evaluative Response 
(student evaluation 
of faculty) 





CORE VALUES 



Evaluative Stimulus 
(evaluation Instrument) 
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Results and Discussion 
Relationship of Values to Student Evaluation of Course and Ingtructor ; 

Valuas ware found to have no significant relationship with student 
avaluatlon of course and Instructor, (See Table I) Apparently the student 
evaluation of course and instructor was not enough to call values into action* 
Student avaluatlon of course and instructor did not appear to be a significant 
function of values* 

Relationship of Adult Education Attitudes of Student_ jyaluatlon of 
Course and Instruc tor i 

The correlation of student evaluation of course and instructor 
to students' adult education addltudes was .2913 which was significant 
at the *01 level* Regression analysis indicated that students' adult edu- 
cation attitudes accounted for eight percent of the variance in students I 
evaluation of course and instructor which was significant at the ^OS 
level* (Table II) The higher the students' total attitude score (toward 
andragogy) the higher his evaluation of the course and the instructor. 

Relationship to Values to Adult Education Attitude ' 

Both regression and correlation analysis Indicated a significant 
relationship between students' values and their adult education attitudes* 
The regression analysis showed that the seven value factors together 
accounted for 26 percent of the variance of adult education attitudes. 
(Table III), 

Of the seven value factors in the regression analysis, and 
proved to be the most Important* The variable had over twice the 
effect on adult education attitudes as did X^ . The variable had a 
negative relationship with adult education at^;*°udes* Further explanation 
of the variable "social versus personal orientation" Indicated that 
the more students valued "A World at Peace, National Security , Equality , 
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Table I. ^alysis of variance for regression of scadints' valih-s from their evaluation of 
course and instructor* 



- 

Source 


DF 


Partial SS 


F 


Value 


probability > F 


^1 


I 




5669 


0 


.08935 


0.7661 




h 
h 
h 
h 




IL 


3624 


1 


.79071 


0.1862 






0, 


1740 


0 


.02743 


0.8690 






0. 


9055 


0 


. 14270 


0.7070 






2. 


9261 


0 


.46115 


0.4998 








0072 


0 


.00113 


0.9733 




h 




4 


2837 


0 


,67511 


0,4147 




Source of 
















Variance 


DF 


SS 


m 


F Value 


Probability > F 


R-Square 


Regression 


7 


30,4686 


4.35 


0.68 


598 


0.6851 


0.0777 


Error 


. 57 


361.6755 


6.35 










Corrected Total 


64 


392. 1441 













S = 65. 
Not slgntflcant. 
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Tabla II. Analysii of variance for regrasslon of studants* adult education additudes from 
thsi" avaluatlon of courie and instructor 



Source of 
Variance 


DF 


SB 


MS 


F Valua 


Probability > F 


R- Square 


Regression 
Error 

Corrected Total 


1 

63 

64 


32.1840 
359,9601 
T9"2a441 


32.184 
5,714 


5.633* 


0.0207 


0.0821 



*N - 65. 



m ,05 level of signif icanee. 
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III. Analysis of variance of reiression of students' value factors from students' 
adult aducatlon attitude" 



Source 


DF Partial SS 


F Value 


Probability >F 


^1 


1 5.8074 


0.4482 


.5056 


^2 


1 32.9592 


2.5437 


,1162 


^3 


1 1L2107 


0,8652 


.3561 


h 


1 45,3737 


3.5091 


.0663 


h 


1 0.3562 


0.0275 


.8689 


h 


1 L9943 


0,1539 


. 6963 




1 0.0003 


0,0001 


.9961 


Source 

of Vartanct 


DF SS MS 


F Value 


Probability >F R=Square 



Ragreasion 7 260,700 37,243 2.874** .012 

Error ^ 751.507 12.957 

Corrected Total 65 1012.207 



% * 65, 

^ ,01 level of significance. 
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Freedom, True Friendship and Self-respect" the higher their orientation 
toward andragogy* 

Table IV presents the correlatlona of the 36 individual values with 
adult education attitudes^ Of the total 36 values, six values were 
significant at the .001 levels one value was signlflca t at the *01 level 
and 11 were significant at the .05 level. Half of the 36 values had a 
significant relationship with students' adult education attitudes. 

The seven values with correlations at the .01 level or batter in 
Table IV indicated that students with an andragogical orientation (high 
score on adult education attitudes) highly valued "Equality, Broadininded , 
Helpful^ and Imaginative ^ " Those same students tended to place low value 
on "A Comfortable Life, Salvation , and Qbedlent." 

The results indicated a strong relationship between students' values 
and their adult education attitudes. 

The Interrelationship of Values and_jMult Education Altitudes with 

Student Evaluation of Course and Instructor ! 

An analysis of variance of regression (Table II) containing two 

value factors (X^ and X, ) and adult education attitudes (Y) accounted for 
2 4 

27 percent of the variance In student evaluation of nourse and Instructor 
which was significant at the ,001 level. 

The path analysis In Figure 2 shows the relative Influence of the 
independent variables ^ Yj Xgs and X^ on the dependent variable Educa- 
tional attitudes (Y) had the strongest Influence on student evaluation 
of course and instructor (Z) with a standardized beta weight of ,506. 
Value factors X^and X^had almost equal Influence on Z with beta weights 
of ^,325 and ,348 respectively. It should be noted that X2 had an 
inverse relationship with Y while X had a positive relationship with Y. 
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Table IV, Correlation MattlK of Values Held by Students 
with Their Adult EdyGatlGn Attitudes and 
Evaiuationof Coursa and Ins cruet or 





Values of Students 


Studc'^^t Adult 


Student Evaluation of 




Education Attitcides 


Coarse and Instructor 


Terminal Values 






A Comfortable Life 


,4366*** 


. 1646 


An Exciting Life 


-,1118 


.0003 


A ^ensa of AcCQmplishment -.1753 


aio9 


A World at Peace 


- » 2495* 


-,0500 


A World of Beauty 


2268* 


-,1901 


EQuali ty 


- . 4133*** 


-.0347 


F^nily Security 


. 2255 


. 2308* 


1? ft a H Ofn 


« i** i y - - 


.0176 


11 B W ^ I S<E 3 ^ 




,1411 


Tnn&r Hs'f'nionv 


. 2540* 


1709 




- 1 ft 1 Q 


-,0901 




, .V J £i J 


-,2377* 






.0236 


Salvation 


3 73 7*** 


-,0202 


Self Respect 


1529 


,1031 


Social Recognition 


- . 0 140 


.0173 


True F^lendahlo 


-^.1125 


,0396 


Wisdom 


-.0933 


-, 1326 


Tns t I'limen t al Values 






Ambitious 


, 2700* 


,2089 


Broadminded 


-,3178* 


-.1439 


Capable 


-,0126 


,0982 


Cheerful 


-.0741 


. 1412 


Clean 


. 2838* 


,0181 


Courageous 


-.2844* 


-,2862* 


Forgiving 


.0512 


-.0857 


Helpful 


-,4280*** 


-,0144 


Honest 


-.0703 


-,0029 


Imaginative 


-,4317*** 


2461* 


Independenc 


-.0281 


, 1329 


Intellectual 


, 1105 


1102 


logical 


. 2447* 


-,1846 


Loving 


-,2170 


-.0227 


Obedient 


. 3827*** 


-,0083 


Polite 


, 275 1* 


.2080 


Responsible 


, 1995 


.3141** 


Self-Controlled 


. 1800 


,0609 


m .05 level of"; 


:^ignif icance. 










e .01 level oy 


significance. 




^ ,001 level of 


significance. 
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Table V 

Analysis of Variance of Re|resalon of Students' Value Factors (X^ and X,) and 
Adult Education Attitudes (Y) from Students' Evaluation of Course and Instructor (Z) 



Seuroe DF . . Artlal SS F Value mbtMll^ > f 

^2 1 43,2849 9.1l?58 ,0035 

1 42,3062 8,9879 ,003a 

Y 1 86,1904 18.3110 ,0001 

Settfea of 

VaFianee DP SS MS F ViuLm ProbtbiUtyaF R-Sqaara K 

Rsgresiisn 3 116,6898 38,9633 8.2777**» ,00% .2734 

aror §6 11Q.6619 4,7070 



Corf«etad Total 69 427,5537 



- ,001 Isval of algnlfieanes. 
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STUDENT 
VALUES 



ADULT STUDENT 
EDUCATION 
ATTITUDES 



STUDENT EVALUATION 
OF COURSE AND 
INSTRUCTOR 




FIGURE 2, PATH ^ALYSIS OF THE RELATIVE EFFECT OF STUDENTS" VALUES 
(Xg MD X.) AND STUDENTS' ADULT EDUCATION ATTITUDES (Y) 
ON STTOENt EVALUATION OF COURSE AND INSTRUCTOR (Z) USING 
STANDARDIZED BETA WEIGHTS* 
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The correlation coefficient between and was only .0002 which was 
nonsignificant. The R-Square value of .273 Indicated that (1 - .273) 
72*7 percent of the variance was not explainad by the model. The R^Square 
of ,214 Indicated that and explained 21 percent of the variance in Y 
and left 79 percent of the variance unexplained. The three variables X^t 
X^, and Y explained more of the variance In Z than and X^ explained In Y, 

The combination of variables X^s X^ and Y explained 27.4 percent of 
the variance in Z. All seven value factors (X^ = X^) accounted for only 
7.7 percent of the variance while adult education attitudes explained 8.2 
percent of the variance* By combining only two value factors and adult 
education attitudes the variance explained In Z was greater than the 
combined sums of both sets of variables independently. The path diagram 
shows clearly that there Is an interaction effect of students' values 
and adult education attitudes on their evaluation of a course and the 
instructor , 

A closer analysis of the Path Model in Figure 1 shows that the X2 
variable operated directly on student evaluation of course and instructor 
while the X^ variable has a stronger "path" through adult education 
attitudes than directly to student evaluation of course and instructor. 
Those students rating the course the highest tended to have andragogical 
attitudes and highly value "Logical^ Imaginatives Intellectual, Independent, 
Forgiving s Balvatiuas Helpful ^ and Clean." 

Sunmary 

The study strongly suggested that student evaluation of course 
and instructor is partially a product of student's values and adult 
education attitudes. Selected student value factors and adult education 
attitudes of students accounted for 27 percent of the variance in their 
racines of their course and instructor. 



It was found chat values by themselves did not significantly Influence 
th©! students' evaluation of their course and the instructors. Students' 
adult education attitudes were shown to have a significant relationship 
with their evaluation of the course and the instructor. The study also 
indicated that students' values were strongly related to their adult 
education attitudes. 

The interacting effect of certain student values factors with adult 
education attitudes to cause variance in the students' evaluation of 
course and instructor was significant to the ,001 level. 

The study showed that the adult education attitudes were more 
closely related to evaluation than values, taong all variables the 
correlation between values and attitudes was the highest. Together 
attitudes and values hr i a strong effect on evaluation. Adult education 
attitudes were significantly related to both values and student evaluation 
of course and instructor. 

Recongnendations for Additional Research 
Based on the review of the literature and the research reported, the 
following recoimnendatlons for further research are suggested i 

1. That the design be repeated with a larger N, a stratified sample, 
an undergraduate populationi and substitute evaluation instruments. 

2, That the amount of attitude or value change occurring during 

a course be considered as a predictor of student evaluations of Instructors, 

3. That the level of congruence between students* and professors' 
attitudes and values be considered as an Indicator of student evaluation 
of course and instructor. 

4, The amount of influence the instructor has an student evaluation 
of course and Instructor be studied. 
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5, The level of congruence between students' and professors' 
parsonaJity characterlstlCH be exaiinned as lu\ indicator of Htudcnt 
evaluation of course and instructor, 

6, That the values and adult education attitudes be repeated 
with a larger population with a mnre complete analysis. 

7, That values and adult education attitudes instruments be 
studied as to their usefulness in predicting success as adult education 
educators. 
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